fix(cli,importers): stop importer warnings from smearing the sync frame - #331
Merged
Conversation
Importers reached for package-level slog, so every warning went to slog.Default() on stderr. During `prosa sync` in a TTY that lands in the middle of a Bubble Tea frame being repainted in place on stdout, which desyncs the renderer's line accounting and smears the output. Give ImportOptions a scoped *slog.Logger, mirroring pusher.logger, and resolve it through importerutil.Logger (nil falls back to slog.Default()). codex and claude-code thread it into parseSession via a closure so ParseFunc keeps its signature and the four importers that never log stay untouched; grok-build passes it down as a parameter because its Importer is a shared singleton. peekSessionID stays silent: RunSingleFile scans the same file twice, so logging there would double-report every bad line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SGJVWRRZLmai3UHR7cHtJ6
Only the pusher's logger was scoped, so importer warnings still hit stderr while the Bubble Tea frame was repainting and smeared it. Point both at the same counting logger from one named step, and let the existing Warnings summary row report the tally for both phases. The plain and JSON paths leave opts.Logger nil, which is what keeps the row's "use --verbose to see them" truthful. slog.SetDefault stays out of this path (#154). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SGJVWRRZLmai3UHR7cHtJ6
Codex appends concurrently, so a torn write leaves a line cut mid-token every few thousand records; today each one earns its own WARN. Count them during the scan and emit a single record per file carrying the tally, the first offending line, and the first error, which is what distinguishes a torn tail from real corruption. claude-code and grok-build get the same treatment. codex and claude-code also skip blank and whitespace-only lines instead of reporting them as malformed, matching what grok-build already did. grok-build's own emptiness guard stays byte-exact: the lines it collects are hashed verbatim into the projection, and that hash is the dedup key, the sync hash, and RawHash at once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SGJVWRRZLmai3UHR7cHtJ6
Record that importer diagnostics travel on opts.Logger, that nothing writes outside the renderer while an in-place frame is live, and that the Warnings summary row now covers both the importers and the catch-up phase. That row was already printed but missing from the summary grammar. Corrects two claims in the CLI architecture page that the same paragraph carried: the interactive decision needs both stdout and stderr to be TTYs, and sync writes its summary to stderr. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SGJVWRRZLmai3UHR7cHtJ6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The report
prosa synccame back with a garbled progress display:Two independent facts produce it.
1. The source data really is torn — the warnings are correct
Codex appends to
sessions/**/rollout-*.jsonlwhile a session runs, and a tornwrite occasionally leaves a line cut mid-token:
Measured across one day's tree: 17 torn lines in 64 041 lines, hitting 15 of
125 files. Every line number in the warning output points at a genuinely
invalid line, and
errors 0confirms the imports themselves succeeded. Not abuffer problem either — the scanner is already at 16 MiB and the longest line
in the worst file is 1.3 MB. No parse decision changes in this PR.
2. Nothing owned the terminal
spinner.Rundrives Bubble Tea on stdout with no alt-screen, so thestandard renderer repaints by moving the cursor up over its own last-frame line
count. Importers called package-level
slog.Warn→slog.Default()→stderr, unbuffered, while that frame was live. Each interleaved write
desyncs the renderer's line accounting.
The mechanism to prevent this already existed but covered only the pusher:
push.loggeris swapped forwarningCounterHandlerand the tally surfaces asthe
Warningssummary row. Importers had no equivalent hook.The fix
ImportOptionsgains a scopedLogger *slog.Logger, resolved throughimporterutil.Logger(opts)(nil →slog.Default()). codex and claude-codethread it into
parseSessionvia a closure, soParseFunckeeps its signatureand the four importers that never log stay untouched; grok-build passes it down
as a parameter because its
Importeris a shared singleton.runSyncInteractivepoints both the pusher and the importers at the same counter, so the existing
summary row reports both phases.
slog.SetDefaultstays out of this path — #154 deliberately removed it, andTestReconcileUsesPusherLoggerlocks that in.Separately, malformed lines are now reported once per file (count + first
offending line + first error) instead of once per line, and codex/claude-code
skip blank lines instead of counting them as malformed. grok-build's own
emptiness guard is left byte-exact on purpose: the lines it collects are hashed
verbatim into the projection, and that hash is the dedup key, the sync hash, and
RawHashat once —TestImportHashUnchangedByBlankLineslocks it.Verified locally
Isolated store (scratch
HOME), 67 real Codex files including all 15 damagedones, run under a pty so the interactive path actually engages.
WARNlines inside the frameWarnings 15 diagnostic logs suppressed in TTY; use --verbose to see themsessions/turns/session_tools/session_usage/kindsid,raw_hash,raw_size,first_prompt)4879c863…4879c863…f850d8f7…f850d8f7…Import output is byte-for-byte unchanged. Also confirmed:
--verboseemits 15 aggregated warnings summing to exactly 17 skippedlines, each carrying
count=andfirst_line=.imported 0 · skipped 67) with zero warnings —RunSingleFileshort-circuits on the hash beforeParse.--jsonstill keeps stdout pure NDJSON with diagnostics on stderr.just ciandjust qualitypass.🤖 Generated with Claude Code
https://claude.ai/code/session_01SGJVWRRZLmai3UHR7cHtJ6